In satellite systems, orthogonal signalling, that ensures absence of intersymbol interference (ISI), is often adopted. For example, in the 2nd-generation satellite digital video broadcasting (DVB-S2) standard [1], a conventional square-root raised-cosine (RRC) pulse shaping filter is specified at the transmitter. In an additive white Gaussian noise channel and in the absence of other impairments, the use of a RRC filter at the receiver and proper sampling ensure that optimal detection (i.e. detection with minimal symbol error rate) can be performed on a symbol-by-symbol basis. On the other hand, it is known that, when finite-order constellations are considered (e.g., phase-shift keying (PSK)), the efficiency of the communication system can be improved by relaxing the orthogonality condition, thus introducing ISI. For example, faster-than-Nyquist signalling (FTN, see [2, 3, 4]) is a well known technique consisting of reducing the spacing between two adjacent transmit pulses in the time-domain well below the Nyquist rate (“Time packing”).
Moreover, the frequency spacing between carriers in multi-carrier transmission can be reduced (“Frequency packing”) to increase spectral efficiency while introducing some inter-carrier interference (ICI).
Even when orthogonality is not given up on purpose, some amount of ISI is introduced by the nonlinear response of the satellite transponder or the ground segment power amplifier (“nonlinear ISI”, as opposed to the “linear ISI” due to Time packing).
Several works in the published literature deal with optimal—or at least nearly-optimal—detection of signal transmitted over linear and/or nonlinear channels with ISI and, when applicable, with ICI. However, the complexity of the receivers of the prior art easily becomes unmanageable.
This is indeed the case for the optimal detector for the “original” FTN signalling, described in [5, 6]. Moreover, these papers do not provide any hint on how to perform the optimization in the more practical scenario where a reduced-complexity receiver is employed.
Paper [7] teaches maximizing the achievable spectral efficiency (ASE) for an additive white Gaussian noise (AWGN) channel, with single or multiple carrier transmission. Detection is performed using a suboptimal symbol-by-symbol detector. Reference [8] describes a more sophisticated detection algorithm with constrained complexity which, however, is still sub-optimal and limited to the case of a linear channel.
The most effective detection algorithms over linear ISI channels, from complexity and performance points of view, are those employing turbo detection [15, 16, 17], which are based on the exchange of information between two (or more) soft-input soft-output (SISO) devices that iteratively refine the quality of their outputs. In DVB-S2, turbo detection can be already considered between the inner detector coping with the channel phase noise and the outer LDPC decoder [18]. Two approaches for maximum-a posteriori (MAP) sequence detection in the presence of linear ISI are known since the early Seventies, due to Forney [19] and Ungerboeck [20]. Both approaches can be extended to MAP symbol detection by resorting to the general forward-backward algorithm (FBA) derived in [21]—see [22] for Ungerboeck's approach. The main drawback of these algorithms is that their trellis size grows exponentially with the number of interfering symbols, so that the implementation of suboptimal algorithms that provide a convenient performance/complexity trade-off is mandatory. Most low-complexity algorithms in the literature maintain the three-stage structure of the FBA, and obtain complexity reduction by performing a simplified trellis search (for instance, see [23, 24]). In particular, the algorithm in [24] works on the trellis describing the channel memory but explores only the most promising paths, chosen according to the MAP criterion. Similarly to all other algorithms in the literature, this algorithm provides a satisfactory performance compared to the optimal detector for a linear channel when the Forney observation model is adopted, but, for reasons discussed in [25, 24], does not perform effectively when the Ungerboeck model is adopted.
The soft interference cancellation (SIC) algorithm proposed in [26, 27] for code-division multiple-access systems can still be applied to the linear ISI scenario. However, its complexity is quadratic in the channel memory when the sliding window approach described in [17, 16] is adopted and cannot extended to the case of a non-linear channel.
A different approach, using the framework based on factor graphs (FGs) and the sum-product algorithm (SPA), was investigated recently [28, 29] for linear channels. It promises to provide a very convenient performance/complexity trade-off, with a complexity which increases linearly with the channel memory [28, 29]. The extension of this algorithm to a nonlinear satellite channel has been recently derived in [13] but its performance is not satisfactory in conditions of strong interference, as resulting e.g. from the adoption of the time-packing technique.